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CRITICAL  SUCCESS  FACTORS  AND  THEIR  APPLICATION  TO 
DOD  WEAPON  SYSTEM  ACQUISITION 

ABSTRACT 

Department  of  Defense  weapon  systems  are  growing  more  eomplex  and 
expensive  and  the  serviees  are  under  inereasing  pressure  from  Congress  to  improve  the 
eost  sehedule  and  performanee  of  their  weapons  programs.  The  purpose  of  this  projeet 
was  to  investigate  and  provide  an  overview  of  issues  within  the  DoD  aequisition  system 
and  provide  information  to  assist  program  managers  in  dealing  with  some  of  the 
ehallenges  they  face.  The  goal  of  this  project  was  to  identify  Critical  Success  Factors  and 
determine  if  they  could  be  applied  to  DoD  acquisition  programs  in  order  to  improve  the 
acquisition  process  and  provide  program  managers  with  a  tool  to  assist  them  with 
managing  a  complex  program. 
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I.  INTRODUCTION 


Over  the  past  half  eentury  weapons  systems  have  grown  increasingly  complex 
and  expensive.  In  2002  the  DoD  spent  $110  billion  to  research,  develop  and  acquire  a 
wide  array  of  weapons  systems. i  In  2005,  weapons  system  procurement  accounted  for 
$144  billion  in  spending  and  by  2009  it  is  estimated  that  the  DoD  will  spend  $185 
billion.2 


2001 

Program 

Cost 

Program 

Cost 

R''A-22  Raptor  aircraft 

$SS.O  billion 

Joint  Strike  Fighter 

$206.3  billion 

DDG-S1  class  destroyer  ship 

Future  Combat  Systems 

$127.8  billion 

Virginia  class  submarine 

Virginia  class  submarine 

$80.4  billion 

C-1 7  Globemaster  airlift  aircraft 

$S1.1  billion 

DDG-S1  class  destroyer  ship 

$70.4  billion 

R''A-18E/F  Super  Hornet  fighter 
aircraft 

$48.2  billion 

RA-22  Raptor  aircraft 

$65.4  billion 

Total 

S20O.8  billion 

Total 

S5S0.0  billion 

Souns:  [iiXi  ((teSi:  (MO  (anali^lsand  pfesenlslom). 

Figure  1.  Total  Cost  of  DoD's  Top  Five  Programs  in  Fiscal  Years  2001  and  2006  (in 
2006  Dollars)(From;  Government  Accountability  Office  [GAO],  GAO-06- 

257T, 2006) 

Figure  1  shows  the  DoD's  top  five  programs  from  2001  to  2006  and  the  associated 
cost  growth  among  them.  The  figure  shows  significant  cost  growth  among  those 
programs  and  shows  that  the  top  five  programs  have  almost  doubled  in  cost  from  $290.8 
billion  to  $550.0  billion  in  the  past  five  years.  In  2006  the  Government  Accountability 
Office  made  the  following  observation: 


1  Government  Aeeountability  Offiee,  Major  Challenges  and  Program  Risks,  GAO-03-98,  2003,  50. 

2  Government  Aeeountability  Offiee,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  3. 
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current  military  operations,  sueh  as  those  in  Afghanistan  and  Iraq, 
eonsume  a  large  share  of  DOD  resourees  and  are  eausing  faster  wear  on 
existing  weapons.  Refurbishment  or  replaeement  sooner  than  planned  is 
putting  further  pressure  on  DOD’s  investment  aceounts. 

At  the  same  time  DOD  is  faeing  these  problems,  programs  are 
eommanding  larger  budgets.  DOD  is  undertaking  new  efforts  that  are 
expeeted  to  be  the  most  expensive  and  eomplex  ever  and  on  which  DOD 
is  heavily  relying  to  fundamentally  transform  military  operations.  And  it  is 
giving  eontraetors  inereased  program  management  responsibilities  to 
develop  requirements,  design  produets,  and  seleet  major  system  and 
subsystem  eontraetors.  Figure  1  shows  that  just  5  years  ago,  the  top  five 
weapon  systems  eost  about  $291  billion  eombined;  today,  the  top  five 
weapon  systems  eost  about  $550  billion.3 

With  the  large  monetary  value  and  implications  for  National  defense,  it  is  easy  to 
see  why  Congress  and  the  DoD  have  plaeed  mueh  emphasis  on  oversight  of  weapon 
system  aequisition.  In  developing  major  weapon  systems,  program  managers  have  the 
daunting  task  of  balaneing  many  factors  that  decide  the  eost,  schedule  and  performanee 
of  their  programs.  There  are  many  aspeets  that  ean  affeet  weapon  system  programs,  both 
positively  and  negatively.  Many  of  these  faetors  that  affeet  a  program  eannot  be 
eontrolled  by  the  program  manager.  Faetors  sueh  as  politieal  turmoil,  ehanges  in  budget 
priorities  and  personnel  ehanges  within  the  DoD  ean  all  have  disastrous  eonsequenees  on 
a  program.  Many  of  these  factors  are  outside  the  seope  of  what  a  program  manager  is 
expeeted  to  manage.  This  research  will  examine  the  program  faetors  in  whieh  the 
program  manager  has  some  control. 

Analyzing  data  from  research  conducted  by  the  GAO  and  other  relevant  sourees 
will  lead  to  the  identifieation  of  key  factors  that  contribute  to  the  suceess  or  failure  of  a 
weapon  system  programs. 

A,  PROJECT  OBJECTIVES 

The  primary  objective  of  this  projeet  is  to  identify  Critieal  Sueeess  Factors  (CSF) 
that  lead  to  the  sueeess  or  failure  of  a  program.  My  intent  is  to  provide  the  aequisition 


3  Government  Aeeountability  Offiee,  DoD  Acquisition  Outcomes-A  Case  for  Change,  GAO-06-257T, 
2006,  3. 
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professional  some  guidelines  to  identify  sueeess  factors  in  managing  complex 
DoD  weapon  system  programs.  If  a  professional  is  aware  of  and  is  able  to  identify  the 
Critical  Success  Factors  that  this  research  identifies,  then  the  acquisition  professional  will 
be  able  to  employ  strategies  and  gain  personal  attributes  that  positively  influence  the 
success  of  future  programs.  By  identifying  both  the  technical  and  educational/experience 
factors  in  DoD  acquisition  programs,  acquisition  professionals  will  be  better  able  to 
improve  the  acquisition  process  in  an  informed  and  realistic  manner. 

B,  RESEARCH  QUESTIONS 

1.  Primary  Research  Question 

What  are  the  Critical  Success  Factors  (CSF)  that  influence  the  successful 
acquisition  of  DOD  weapon  system  programs? 

2.  Secondary  Questions 

•  How  do  we  define  the  success  of  an  acquisition  program? 

•  What  aspects  lead  to  the  successful  acquisition  of  a  DOD  weapon  system? 

C,  SCOPE  OF  PROJECT 

This  research  will: 

•  Identify  Critical  Success  Factors  that  are  applicable  to  DoD  acquisition 
programs 

•  Provide  DoD  acquisition  professionals  an  identification  of  Critical  Success 
Factors  that  have  been  proven  to  positively  influence  the  success  of  DoD 
major  acquisition  programs. 

D,  ORGANIZATION 

Chapter  II  will  discuss  the  background  of  program  management  in  the  DoD  as 
well  as  the  history  of  weapons  system  development.  This  chapter  will  also  examine  the 
major  changes  that  have  affected  the  acquisition  process.  It  will  also  address  the  parties 
involved  in  the  process  that  influence  the  outcome  of  programs. 

Chapter  III  will  present  data  on  Critical  Success  Factors  that  affect  a  program. 
The  research  will  focus  on  research  published  by  acquisition  professionals  and  the 
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Government  Aeeountability  Offiee  (GAO)  as  well  as  past  weapons  programs  and  analyze 
factors  and  events  within  those  programs.  This  project  will  identify  factors  that  had  a 
major  effect  on  the  success  or  failure  of  the  program. 

Chapter  IV  analyzes  the  data  presented  from  Chapter  3  and  compares  that  data  to 
identify  any  correlations  between  the  research  that  has  been  conducted  the  GAO  reports 
and  any  other  evidence. 

Chapter  V  presents  the  conclusions  and  the  recommendations  regarding  program 
managers  and  their  programs. 
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II.  BACKGROUND 


During  World  War  II  the  acquisition  of  weapons  systems  was  a  relatively  simple 
process  compared  to  today.  During  that  period,  a  need  was  identified,  a  contractor  chosen 
and  a  product  was  produced,  usually  in  very  large  quantities.  The  weapons  were 
relatively  simple  to  produce  by  today's  standards,  and  that  simplicity  resulted  in  weapon 
systems  that  could  be  produced  in  large  quantities  by  companies  that  had  little  experience 
manufacturing  weapons.  For  instance,  during  WWII  roughly  50,000  M4  "Sherman"  tanks 
were  produced  for  the  US  and  its  allies.^  These  tanks  were  relatively  simple  and  easy  to 
produce  in  large  numbers.  By  contrast,  the  Ml  "Abrams"  series  tank  has  only  been 
produced  in  quantities  of  about  8,800  over  a  twenty  five-year  period.^  The  Ml  tank 
contains  computerized  fire  control  systems  and  other  digital  systems  as  well  as  software 
that  have  greatly  increased  the  capability  of  each  tank.  This  capability  is  why  the  US 
forces  are  the  best  in  the  world,  but  with  the  increased  capability  comes  increased 
complexity.  The  same  increase  in  complexity  we  see  in  tanks  can  be  seen  in  other 
systems  as  well,  including  aircraft.  Currently,  the  F22  and  FI 8  aircraft  are  the  premier 
fighter  aircraft  in  the  US  arsenal.  These  aircraft  use  computers  to  control  flight  control 
surfaces  in  what  is  known  as  fly-by-wire.  These  systems  are  very  complex  compared  to 
the  planes  that  were  flying  even  20  years  ago  and  these  complex  weapon  systems  are 
being  produced  in  decreasingly  smaller  quantities.  As  Figure  2  illustrates,  weapon 
systems  are  growing  in  cost  and  as  the  cost  grows  the  buying  power  is  reduced  resulting 
in  fewer  items  being  purchased. 


^  R.P.  Hunnicutt.  Patton:  A  History  of  the  American  Main  Battle  Tank.  1984,  Presidio  Press,  65. 

^  http://www.fas.org/man/dod-lQl/svs/land/ml.htm.  Ml  Abrams  Tank  Production,  01  December  2006. 
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Program 

Initial  Initial  Latest  Latest  Percent  of  unit 

estimate  quantity  estimate  quantity  cost  increase 

Joint 

Strike 

Fighter 

$189. 8  billion  2.866  aircraft  $206.3  billion  2.458  aircraft  26.7 

Future 

Combat 

Systems 

« 

Olf'  ’  ■ 

♦ 

$82.6  billion  15  systems  $127.5  billion  15  systems  54.4 

F-22A 

Raptor 

$81.1  billion  648  aircraft  $65.4  billion  181  aircraft  188.7 

Evoived 

Expendable 

Launch 

Vehicle 

$15.4  billion  181  vehicles  $28.0  billion  138  vehicles  137.8 

■A 

Space 

Based 

Infrared 
System  High 

$4.1  billion  5  satellites  $10.2  billion  3  satellites  315.4 

Expeditionary 

Fighting 

Vehicle 

$8.1  billion  1.025  vehicles  $11.1  billion  1 ,025  vehicles  35.9 

Si^ce:  GmO  enaK'Sts  or  DOO  data,  «ourced  in  tret  r«6pactn<i»  orcter:  JSF  r^ogram  Offlcac  Rogram  Manage*.  UMI  or  Acticn. 

U.&  army:  F-22A  Sysiant  Rogram  Onioa;  iLali)  02003  lL&l.c<ar>aed  Marorx  (Tlghit  0  2003  TTm  Soerng  Oimpany:  Locat>»ad  Marlin 
Spaca  Syetama  Company;  G«r»af8i  Dynamica  Land  SyaianK 

Figure  2.  Examples  of  DoD  Programs  with  Redueed  Buying  Power  (2006)  (From; 

GAO,  GAO-06-585T,  2006) 


As  Figure  2  illustrates  as  the  cost  of  development  and  production  go  up  the  services  make 
trade-offs  that  lead  to  reduced  numbers  of  systems  purchased.  When  the  services  decide 
to  by  fewer  systems  the  cost  per  unit  goes  up  dramatically  because  the  contractors  fixed 
costs  are  spread  over  fewer  units. 

The  increasing  complexity  of  weapons  systems  along  with  the  ever  increasing 
demand  on  the  federal  budget  has  put  enormous  pressure  on  DoD  acquisition  programs. 
As  the  complexity  of  weapons  has  gone  up  so  has  the  cost  of  the  weapons.  This 
complexity  of  weapons  systems  also  causes  the  increase  in  development  time.  The 
increases  to  both  the  cost  and  schedule  must  be  balanced  with  the  performance  of  the 
weapon.  How  much  performance  can  we  afford?  How  long  will  it  take  to  develop  the 
technology  to  give  us  that  level  of  performance?  The  answers  to  these  questions  provide 
the  trade-offs  that  must  be  made  in  any  program. 
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A,  THE  STAKEHOLDERS 

There  are  many  people  and  organizations  that  influence  the  outcome  of  a  DoD 
program.  To  understand  the  environment  program  managers  must  work  within  we  have 
to  first  understand  the  people  and  organizations  that  influence  their  programs.  The 
diagram  below  shows  some  of  the  most  influential  people  and  organizations. 


The  tortured  triangle  shows  the  three  main  groups  that  have  varying  degrees  of 
influence  over  a  program  and  how  the  groups  are  intertwined  and  try  to  exert  power  over 
one  another.  The  defense  industry  exerts  influence  through  the  lobbying  of  members  of 
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congress  and  the  executive  braneh  using  taetics  sueh  as  campaign  contributions  and 
political  favors  in  order  to  influence  both  the  executive  braneh  and  eongress  to  win 
support  for  programs  that  produee  weapon  systems  developed  by  the  eontractor.  The 
executive  branch  exerts  its  power  through  control  of  the  DoD  and  the  services.  The 
exeeutive  braneh  (the  serviees  and  DoD)  sets  its  priorities  through  the  official  policies 
and  strategies  (National  Military  Strategy,  National  Seeurity  Strategy,  Quadrennial 
Defense  Review,  etc.)  and  the  budget  process  (Planning  Programming  Budgeting 
Exeeution  System,  Future  Years  Defense  Planning  Guidanee,  Program  Objectives 
Memorandum,  etc.).  The  executive  branch  also  exerts  influence  by  appointments  and  the 
plaeement  of  key  personnel  to  oversee  the  process  within  the  DoD  and  the  services. 
While  it  is  called  the  Presidents'  budget,  without  Congressional  approval,  no  funding  is 
authorized  or  alloeated  for  the  programs.  This  is  the  main  eontrol  that  Congress  has  for 
influeneing  weapon  system  aequisition.  With  the  ever  inereasing  eomplexity  and  cost  of 
weapon  systems  Congress  has  eontinued  to  try  to  exert  more  control  over  the  aequisition 
proeess.  Congress  has  eontinually  called  for  ehanges  in  the  way  weapons  systems  are 
aequired.  In  an  attempt  to  control  spending.  Congress  keeps  a  close  watch  on  the  DoD 
and  weapon  system  programs.  The  following  ehart  shows  the  weapon  systems  cost 
overruns  faced  by  the  DoD  since  the  1970s  and  the  initiatives  made  by  Congress,  the 
Exeeutive  Branch  and  DoD  to  address  problems  with  the  aequisition  proeess. 
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1970-1978 

1980  -1989 

1990-1999 

Development 
cost  ovenun: 

$13  billion 

(30%) 

Development 
cost  overrun: 

$12  billion 

(39%) 

Development 
cost  overrun: 

$15  billion 

(40%) 

K«y  StudiM  and  IrritlatlvM  Impeding  th»  Dtfcntt  Aoqiitition  ProoMft 

•  1970  Fitzhugh  Commission 

•  1972  Commission  on  Government 
Procurement 

•  1981  Carlucci  Initiatives 

•  1982  Grace  Commission 

1986  Packard  Commission 

•  1994  Federal  Acquisition  Streamlining  Act 

•  1996  Clinger-Cohen  Act 

DOD  Acquisition  Policy  ChangM 

•  1971  DOD  5000  policy  established 

•  1975  Policy  revised 

•  1977  Policy  revised 

•  1980  Policy  revised 

•  1982  Policy  revised 

•  1985  Policy  revised 

•  1986  Policy  revised 

•  1987  Policy  revised 

•  1991  Policy  revised 

•  1996  Policy  revised 

Soifce:  [X>D  (data);  GAO  (analy’ste  andpr^^entadio). 


Figure  4.  Development  Cost  Overruns  by  Decade  (in  Fiscal  Year  2005  Dollars)  and 
Key  Reform  Efforts.  (From:  GAO,  GAO-06-368,  2006) 


The  Congressional  budget  process  and  the  changes  to  priorities  often  result  in 
cancellations  or  reductions  in  program  budgets,  which  result  in  inefficiencies  in  the 
procurement  process.  When  fewer  defense  dollars  are  available,  decisions  must  be  made 
and  the  results  are  usually  fewer  weapons  purchased  over  a  longer  period  of  time.  This 
causes  an  increase  to  the  per-unit-cost  of  each  item  and  causes  contractor's  costs  to  go  up 
because  they  are  producing  an  inefficient  quantity  of  weapons. 

The  stakeholders  listed  above  can  have  varying  degrees  of  influence  over  a  DoD 
weapons  program,  based  on  many  factors.  This  research  describes  these  organizations 
and  individuals  to  depict  the  many  factors  that  can  influence  a  program  and  are  outside 
the  control  of  the  PM.  There  are  also  many  problems  that  affect  a  program  from  unstable 
funding  to  changes  in  defense  priorities.  These  problems  are  usually  beyond  the  scope  of 
the  program  manager.  The  program  manager  must  focus  on  what  he  can  control  in  order 
to  manage  a  successful  program.  This  project  will  focus  on  the  factors  that  the  PM  can 
control  and  determine  which  of  those  factors  influence  the  success  or  failure  of  the 
program. 
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B,  PROGRAM  MANAGEMENT 

What  is  program  management?  Program  management  is  the  proeess  by  whieh  the 
DoD  develops  current  weapon  systems.  The  DAU  glossary  defines  program  management 
as: 


The  process  whereby  a  single  leader  exercises  centralized  authority  and 
responsibility  for  planning,  organizing,  staffing,  controlling  and  leading 
the  combined  efforts  of  participating/assigned  civilian  and  military 
personnel  and  organizations,  for  the  management  of  a  specific  defense 
acquisition  program  or  programs,  through  development,  production, 
deployment,  operations,  support  and  disposal.^ 

The  concept  of  program  management  can  be  traced  to  the  Air  Force's  ballistic  missile 
development  in  the  1950  and  1960s  and  the  Air  Force  Systems  Command's  development 
of  publications  and  manuals  referred  to  as  the  375  series.  In  Acquiring  Defense  Systems, 
David  Acker  states  that  program  management  usually  contain  five  distinct  phases: 

(a)  exploration  and  development  of  defense  system  concepts  based  upon  a 
recognized  mission  element  need;  (b)  demonstration  and  validation  of 
selected  alternative  concepts;  (c)  design,  development,  limited  production, 
test  and  evaluation;  (d)  production;  and  (e)  Service  deployment, 
operational  support  as  well  as  planned  and  unplanned  product 
improvement.^ 

This  process,  if  executed  properly  ensures  that  every  aspect  of  the  weapon  system 
from  production  to  support  to  disposal  has  been  identified  and  addressed  to  ensure  the 
success  of  the  system.  While  the  process  of  program  management  is  a  complex  process 
that  produces  a  unique  product,  lessons  can  be  learned  from  previous  programs  and 
implemented  effectively  in  new  programs.  The  majority  of  DoD  acquisition  programs 
experience  unanticipated  increases  to  both  the  cost  and  schedule  of  the  programs.  This 
research  will  identify  internal  factors  from  previous  programs  that  if  applied  to  future 
programs  may  help  to  reduce  the  incident  of  cost  and  schedule  overruns.  While  the 
impact  of  the  decisions  made  by  the  three  groups  of  stakeholders  outside  the  program  has 


^  Defense  Acquisition  University,  Glossary:  Defense  Acquisition  Acronyms  and  Terms,  2005,  22. 

^  David  D.  Acker,  Acquiring  Defense  Systems,  A  Quest  for  the  Best,  Defense  Systems  Management 
College  Press,  1993. 
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a  significant  effect  on  the  "success"  of  a  program,  the  PM  has  little  eontrol  over  them.  On 
the  other  hand,  the  internal  factors  can  be  controlled  by  the  PM. 

C.  DEFINING  SUCCESS 

How  do  we  define  the  success  of  a  program?  This  question  is  an  important  one  to 
consider  when  evaluating  the  cost,  schedule  and  performance  of  a  program.  Can  a 
program  be  successful  if  it  is  completed  over  budget?  What  if  it  doesn't  have  the  level  of 
performance  that  was  required  by  the  user?  There  have  been  multiple  studies  conducted 
by  GAO,  the  DoD  and  other  ageneies  but  most  fail  to  define  the  meaning  of  success. 
While  the  GAO  makes  the  elaim  that  program  managers  use  a  formal  definition  for 
suecess,  when  it  comes  to  budgets  and  funding  the  definition  becomes  ambiguous.  A 
GAO  report  made  the  following  observation: 

At  DoD,  suecess  is  often  formally  defined  in  similar  terms  as  the 
commercial  world:  deliver  high  quality  produets  to  customers  (the 
warfighter)  at  the  right  time  and  the  right  cost.  Virtually  all  program 
managers  we  spoke  with  first  defined  success  in  terms  of  enabling 
warfighters  and  doing  so  in  a  timely  and  eost-efficient  manner.  But  when 
the  point  was  pursued  further,  it  became  clear  that  the  implied  definition 
for  success  in  DoD  is  attracting  funds  for  new  programs,  and  keeping 
funds  for  ongoing  programs.  Program  managers  themselves  say  they 
spend  enormous  amounts  of  time  retaining  support  for  their  efforts  and 
that  their  focus  is  largely  on  keeping  funds  stable.  They  also  observe  that 
the  DoD  starts  more  programs  than  it  can  afford  to  begin  with,  which 
merely  sets  the  stage  for  competition  and  resulting  behaviors.  As  noted 
earlier,  there  are  factors  that  contribute  to  how  success  is  defined  in 
practice,  including  the  fact  that  DoD  depends  on  annual  appropriations 
and  it  must  fund  a  wide  variety  of  missions  beyond  weapon  systems 
development.^ 

Morris  and  Hough  addressed  the  topic  of  project  success  in  The  Anatomy  of  Major 
Projects  and  they  commented:  "Success  is  a  slippery  concept  to  measure-and  that  it  has 
different  definitions  depending  on  who  you  are  and  what  your  role  in  the  project  is  (and 
when  you  attempt  to  measure  it).^  You  can  be  assured  that  the  end  user's  definition  of 
suecess  will  be  different  than  the  PM's.  While  the  PM's  criteria  and  user's  criteria  for 


°  GAO,  Best  Pmctices-Better  Support  of  Weapon  System  Program  Managers  Needed  to  Improve 
Outcomes,  GAO-06-110,  2006,  56. 

9  Peter  Morris  &  George  Hough,  The  Anatomy  of  Major  Projects:  A  Study  of  the  Reality  of  Project 
Management,  1987,  Wiley,  96. 
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success  might  have  some  overlap  the  end  user  doesn't  eare  about  aequistion  eost  but 
wants  a  funetional,  reliable  pieee  of  equipment  that  meets  his  needs  and  he  wants  it 
immediately.  The  PM  has  the  diffieult  job  of  transforming  the  user's  needs  into  the 
requirements  and  attempt  to  balanee  those  needs  with  the  eost  and  sehedule.  The  eost, 
sehedule  and  performanee  will  all  be  impaeted  by  the  ehoiees  that  are  made. 

One  of  the  major  issues  that  this  projeet  will  address  is  that  the  serviees  the  DoD, 
Congress  and  the  GAO  do  not  have  a  definition  of  sueeess  that  is  agreed  upon  and  used 
synonymously.  If  the  organizations  ean  not  agree  on  what  defines  sueeess,  then  how  ean 
it  be  aehieved? 

In  Introduetion  of  Defense  Aequisition  Management  the  author  deseribes  sueeess 
and  what  it  means  to  the  four  major  players  in  the  PM's  environment: 

A  sueeessful  system  aequisition  program  plaees  a  eapable  and  supportable 
system  in  the  hands  of  a  user  when  and  where  it  is  needed,  and  does  so 
within  the  bounds  of  affordability.  The  ideal  outeome  neeessary  for 
sueeessful  long-term  relationships  among  the  partieipants  in  defense 
aequisition  is  "Win-Win",  wherein  eaeh  partieipant  gains  something  of 
value  for  partieipating.  Depending  on  your  perspeetive,  "sueeess"  ean  take 
many  forms. 

•  For  the  PM,  sueeess  means  a  system  that  is  delivered  on  time, 
within  eost  and  meets  the  user's  requirements. 

•  For  the  Offiee  of  the  Seeretary  of  Defense  (OSD)  staff,  sueeess 
means  a  program  that  satisfies  national  seeurity  objeetives, 
provides  a  balaneed  foree  strueture,  and  does  not  attraet  undue 
Congressional  serutiny. 

•  For  industry,  sueeess  means  a  program  that  provides  a  positive 
eash  flow  and  a  satisfaetory  return  on  investment,  and 
preserves  the  eontraetor's  eompetitive  position  in  the  industry. 

•  For  the  user,  sueeess  means  a  system  that  is  effeetive  in  eombat 
and  easy  to  operate  and  maintain. 
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To  a  large  extent,  a  person's  (or  organization's)  perspeetive  on  what  eonstitutes  a 
sueeessful  program  depends  on  position.  In  other  words,  where  you  stand  on  "suceess"  is 
largely  a  funetion  of  where  you  sit.io 

For  every  stakeholder  in  the  aequisition  proeess,  "suecess"  eould  have  a  distinet 
and  different  meaning.  How  ean  we  expeet  to  produee  suoeessful  programs  when  we  ean 
not  even  agree  on  the  definition  of  sueeess?  Although  it  is  diffieult  to  define  a  sueeessful 
program  in  a  way  that  would  satisfy  every  stakeholder,  it  is  an  important  question  that 
must  be  addressed  in  order  to  ereate  a  more  efficient  acquisition  process. 

D,  CRITICAL  SUCCESS  FACTORS 

The  theory  of  "Success  Factors"  was  first  developed  by  D.  Ronald  Daniel  in  1961, 
but  it  was  not  until  1979  that  J.F.  Rockart  of  the  Harvard  Business  School  published  work 
on  a  systematic  approach  to  Critical  Success  Factors  (CSF)  in  "A  Primer  on  Critical 
Success  Factors".!  1  His  work  focused  on  factors  within  organizations  that  if  identified 
and  measured  can  produce  successful  results.  Rockart  defined  CSF: 

...the  limited  number  of  areas  in  which  results,  if  they  are  satisfactory, 
will  ensure  successful  competitive  performance  for  the  organization.  They 
are  the  few  key  areas  where  things  must  go  right  for  the  business  to 
flourish.  If  results  in  these  areas  are  not  adequate,  the  organization's  efforts 
for  the  period  will  be  less  than  desired. 

While  Rockarfs  research  focused  on  identification  of  CSF  at  the  executive  level. 
Dr.  James  Dobbins  and  Dr.  Richard  Donnelly  published  research  in  1998  regarding  CSF 
within  government  acquisition  programs. Their  work  published  in  Acquisition  Review 
Quarterly  focused  on  identification  of  CSFs  within  government  acquisition  programs  that 
can  help  PMs  manage  successfully.  Dr.  Dobbins  published  several  more  articles 
regarding  CSFs  and  their  application  within  the  DoD  acquisition  process  and  risk 
management. 


!!!  Defense  Acquisition  University,  Introduction  to  Defense  Acquisition  Management,  1999,  7. 

! !  James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  62. 

!2  James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  62. 

13  Ibid. 
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E,  SUMMARY 

This  chapter  has  briefly  deseribed  the  organizations  and  people  assoeiated  with 
the  DoD  aequisition  proeess.  It  has  also  demonstrated  the  external  issues  that  the  PM 
must  deal  with  while  managing  a  weapon  system  program.  Sinee  World  War  II  weapon 
systems  have  beeome  more  eomplex  and  thus  more  expensive.  In  order  to  manage  those 
programs  the  serviees  assigns  a  PM  to  manage  the  weapon  system  programs.  The  PM 
must  deal  with  three  main  organizations  or  groups  (Exeeutive,  Congress,  Industry) 
throughout  the  life  of  the  program.  There  are  many  faetors  that  affect  a  program  both 
externally  and  internally.  The  subsequent  chapters  will  identify  the  internal  factors  that 
can  have  a  negative  affeet  on  weapon  system  programs.  There  are  many  ehallenges  that 
faee  the  Program  Manager  of  a  major  weapon  system  program.  He  is  faeed  with  a  system 
that  is  extremely  eomplex  to  produee.  He  is  also  faeed  with  a  multitude  of  people  and 
organizations  that  have  input  into  his  system  and  the  PM  must  try  to  exert  some  eontrol 
over  these  people  and  organizations.  Unlike  the  eommereial  world,  PMs  within  the  DoD 
exert  far  less  eontrol  over  their  programs.  The  next  ehapter  will  foeus  on  the  researeh  that 
has  been  eonducted  regarding  Critieal  Sueeess  Faetors  by  Dobbins  and  others  and  its 
applieation  to  DoD  weapon  aequisition  programs. 
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III.  RESEARCH  ON  SUCCESS  FACTORS 


A.  DOBBINS’  AND  DONNELLY’S  RESEARCH 

In  1998  James  Dobbins  and  Richard  Donnelly  published  research  regarding 
Critical  Success  Factors  and  the  implications  on  government  acquisition  programs.  Prior 
to  their  work,  most  research  on  CSFs  was  conducted  on  for  profit  businesses  in  the 
private  sector.  This  research  focused  on  the  strategic  level,  namely  executive  level 
managers  within  specific  industries.  The  basic  premise  of  Dobbins'  and  Donnelly's  work 
was  to  determine  if  it  was  possible  to  identify  general  CSFs  that  were  relevant  to  most 
major  government  acquisition  programs.  Dobbins  and  Donnelly  sought  to  answer  the 
following  questions,  "Are  there  any  general  Critical  Success  Factors  for  DoD 
programs?"  They  surmised  that  the  identification  of  CSFs  that  are  common  to  DoD 
acquisition  programs  and  the  identification  of  metrics  to  correctly  measure  the  CSFs  will 
help  future  PMs  manage  their  programs  with  a  higher  degree  of  success. 

Dobbins  and  Donnelly  used  a  survey  that  was  developed  based  on  the  CSF 
categories  originally  identified  by  Bullen.i^  The  survey  asked  DoD  PMs  to  identify  CSFs 
in  the  related  categories  and  the  associated  metric  for  the  identified  CSF.  The  survey  was 
mailed  to  two  separate  groups  of  PMs.  The  first  group  consisted  of  PMs  managing 
embedded  system  programs  and  the  second  group  was  PMs  managing  automated 
information  systems.  Dobbins  and  Donnelly  distributed  the  surveys  to  both  groups 
including  73  to  the  embedded  system  PMs  and  57  to  the  automated  information  system 
PMs.  They  received  back  20  completed  surveys  from  the  embedded  system  PMs  and  14 
from  the  automated  information  system  PMs.  The  relatively  low  percentage  of  returned 
surveys,  27%  and  25%  respectively,  is  typical  for  surveys  sent  to  PMs  due  to  the  many 
official  and  unofficial  questionnaires  in  which  PMs  are  asked  to  respond. 

The  results  showed  18  CSFs  that  were  common  to  both  groups  of  PMs.  The 
following  three  charts  show  the  results  for  the  two  groups  and  the  18  combined  CSFs  that 


James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  61. 

15  Ibid.,  63. 
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were  identified.  The  top  four  CSFs  are  highlighted  and  will  be  analyzed  further  in 
Chapter  4.  It  is  interesting  to  note  is  that  the  program  managers  speeifieally  mentioned 
establishing  systems  engineering  expertise  within  the  program  office  as  it  related  to 
having  a  technically  competent  program  staff, 


CSF 

Times 

N  =  20  Factor 

12 

15 

Continuous  meaningful  visibility  using  measures 

4 

9 

Technically  competent  program  office  staff 

2 

9 

Clearly  defined  and  stable  requirements 

1 

8 

Stable  and  adequate  funding 

3 

5 

Risk  management 

7 

5 

Schedule  management 

15 

5 

Stable,  qualified  industrial  base 

17 

5 

Effective  vertical  a  lateral  communication 

16 

4 

Management  political  influencing  agents 

6 

3 

Stable  and  adequate  personnel  resources 

8 

3 

Cost  management 

9 

3 

User  involvement,  support  and  acceptance 

10 

3 

Strong  and  structured  quality  control 

11 

2 

Clearly  and  objectively  defined  project  goals 

19 

2 

Development  and  execution  of  program  management  strategic  plan 

22 

2 

Change  management 

5 

1 

Configuration  management  and  control 

13 

1 

Other  agency  support  for  training  and  government  furnished 

equipment  (GEE) 

14 

1 

Adequate  program  office  resources 

18 

1 

Eeadership 

20 

1 

Thorough  system  documentation 

21 

1 

Test  and  evaluation  master  plan  (TEMP)  approval 

23 

1 

Program  office  teamwork 

24 

1 

Effective  and  timely  decision  making 

25 

1 

Eoreign  military  sales 

26 

1 

Measure  and  control  integrated  logistics  support  performance 

27 

1 

Initiation  of  new  projects 

Table  1 .  CSFs  Identified  by  Embedded  Systems  Program  Managers  in  Priority  Order 
(From:  Summary  Research  Report  on  Critical  Success  Factors  in  Federal 
Government  Program  Management,  1998) 


James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  68. 
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CSF  1 

Times 

1  N  =  14  Factor 

1 

10 

stable  and  adequate  budget 

9 

9 

User  involvement  and  support 

12 

9 

Effeetive  teehnieal  performanee  evaluation 

2 

8 

Detailed  requirements  analysis 

4 

8 

Tecbnically  competent  staff 

19 

7 

Top  management  support 

17 

6 

Effeetive  vertieal  a  lateral  eommunieation 

7 

6 

Sehedule  management 

10 

6 

Strong  quality  eontrol  program 

6 

5 

Stable  projeet  staff 

16 

5 

Management  politieal  influeneing  agents 

13 

4 

Other  ageney  support  for  training  and  government  furnished  equipment  (GEE) 

3 

3 

Risk  management 

20 

3 

Strong  knowledge  of  life  eyele  management 

23 

3 

Ineremental  aequisition 

8 

2 

Cost  management 

22 

2 

Common  Sense 

11 

2 

Clearly  defined  mission 

14 

2 

Adequate  program  offiee  resourees 

21 

1 

Objeetive  eeonomie  analysis 

18 

1 

Leadership 

15 

1 

Stable,  qualified  industrial  base 

5 

1 

Configuration  management  and  eontrol 

24 

1 

On-site  team  to  prevent  fraud,  waste  and  abuse 

Table  2.  CSFs  Identified  by  Automated  Info  Systems  Program  Managers  in  Priority 


Order  (From;  Summary  Researeh  Report  on  Critioal  Sueeess  Faetors  in 
Federal  Government  Program  Management,  1998) 


CSF 

Times 

1  N  =  34  Factor 

12 

24 

Continuous  meaningful  visibility  using  measures 

1 

18 

Stable  and  adequate  funding 

18 

2 

Leadership 

2 

17 

Clearly  defined  and  stable  requirements 

4 

17 

Tecbnically  competent  program  office  staff 

9 

12 

User  involvement,  support  and  aeeeptanee 

7 

11 

Sehedule  management 

17 

11 

Effeetive  vertieal  a  lateral  eommunieation 

10 

9 

Strong  and  stmetured  quality  eontrol 

16 

9 

Management  politieal  influeneing  agents 

3 

8 

Risk  management 

6 

8 

Stable  and  adequate  personnel  resourees 

15 

6 

Stable,  qualified  industrial  base 

13 

5 

Other  ageney  support  for  training  and  government  furnished  equipment  (GEE) 

8 

5 

Cost  management 

11 

4 

Clearly  and  objeetively  defined  projeet  goals 

14 

3 

Adequate  program  offiee  resourees 

5 

2 

Configuration  management  and  eontrol 

Table  3.  Combined  CSFs  Identified  by  Both  Groups  of  Program  Managers  (From: 

Summary  Researeh  Report  on  Critioal  Sueeess  Faetors  in  Federal  Government 

Program  Management,  1998) 
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Dobbins  and  Donnelly  concluded  that  CSFs  in  DoD  acquisition  programs  were 
identifiable  and  published  the  following  findings: 

•  The  CSFs  for  DOD  program  management  are  identifiable,  and 
their  explicit  identification  would  clearly  assist  the  program 
managers  in  maintaining  management  focus  on  the  factors  most 
important  to  program  success. 

•  A  significant  number  of  CSFs  are  common  to  both  groups  of 
program  managers. 

•  The  component  assumptions  and  emphasis  for  a  given  CSF 
common  to  both  groups  may  be  slightly  different.  This  difference 
is  largely  a  function  of  the  difference  between  the  mission  of  the 
two  groups,  the  embedded  system  PMs  being  more  concerned  with 
the  complete  development  of  total  systems  than  the  automated 
information  system  PM. 

•  The  CSFs  identified  by  the  PMs  as  the  most  significant  for 
program  success  are  not  those  factors  that  received  the  most 
attention  from  the  oversight  activities  and  agencies. 

•  The  measures  identified  most  often  by  the  program  managers  as 
those  used  or  recommended  for  use,  are  significantly  more  oriented 
toward  cost  and  schedule  (which  must  be  briefed  to  oversight 
agencies)  rather  than  toward  factors  identified  by  the  program 
managers  in  the  field  as  being  most  critical  to  the  program  success. 

•  There  is  no  widely  recognized  and  generally  used  set  of  measures 
consistent  with  the  most  frequently  reported  CSFs.  This  suggests 
that  even  though  various  factors  are  recognized  as  critical,  they  are 
not  usually  explicitly  identified  and  the  information  network 
required  to  manage  against  those  critical  factors  is  not  well 
developed. 

•  A  commonly  recognized  set  of  CSFs,  and  a  consistent 
measurement-based  information  network  based  on  these  CSFs, 
would  be  of  significant  benefit  to  the  program  managers  as  well  as 
the  oversight  agencies.  Such  a  management  system  would 
significantly  improve  the  management  success  potential  on 
programs  across  the  board,  and  would  provide  the  external  groups 
a  consistent  method  for  evaluating  and  comparing  different 
programs  so  that  recommendations  for  future  improvements  could 
be  results-based. 

•  A  CSF-based  information  network  for  program  management 
would  lend  itself  to  not  only  increased  visibility  for  the  program 
manager  and  staff  during  all  life  cycle  phases,  but  would  provide 
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the  base  for  the  establishment  of  measures  for  determining  when 
the  underlying  assumptions  for  a  given  CSF  may  be  ehanging. 

•  A  CSF-based  information  network  would  provide  a  eommon 
framework  for  produetive  diseussions  between  the  program 
manager  and  the  external  groups,  ineluding  the  development 
eontractors  and  would  greatly  support  the  effeetiveness  of  IPTs. 

•  A  CSF-based  information  network  would  signifioantly  reduee  the 
duplieative  reporting  and  diversions  the  program  managers 
experienee  under  the  present  eonditions. 

•  A  CSF-based  analysis  proeess  would  be  signilieant  teaehing 
instrument  for  edueating  prospeetive  program  managers  in 
strategie  thinking  in  terms  of  those  considerations  critical  to 
success. 1’^ 

Other  research  and  studies  relating  to  the  DoD  have  added  to  the  body  of  knowledge 
regarding  CSFs  and  their  importance  to  DoD  weapon  system  acquisitions. 

B,  GAO-06-110  BEST  PRACTICES 

The  GAO  published  data  regarding  better  support  of  weapons  systems  in  the  best 
practices  series  titled  "Better  Support  of  Weapon  System  Program  Managers  Needed  to 
Improve  Outcomes"  (GAO-06-110).  This  GAO  report  used  the  case  study  methodology 
to  identify  the  best  practices  and  processes  used  by  commercial  companies  to  ensure  their 
programs  were  successful.  The  GAO  selected  companies  that  produced  complex  products 
using  program  management  to  conduct  research,  development  and  production.  The  GAO 
studied  three  companies;  Toyota,  Siemens  and  Motorola,  conducting  interviews  with 
senior  leaders  and  program  managers  to  identify  the  processes,  practices  and  metrics  that 
were  used  by  senior  management  to  support  program  managers  and  hold  them 
accountable  for  their  program.  In  addition,  the  GAO  conducted  a  survey  of  current  PMs 
of  ACAT  I  and  II  programs.  The  survey  was  sent  to  185  PMs  of  which  128  responded 
resulting  in  a  69  percent  response  rate.  The  GAO  also  conducted  in-depth  interviews  with 
individual  PMs  and  Program  Executive  Officers  as  well  as  PMs  from  Boeing  and 


James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  77. 
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Lockheed  Martin  managing  two  major  weapon  systems.  They  also  synthesized 
information  from  previous  GAO  work  regarding  best  practiees  in  product  development, 

The  GAO  found  many  differenees  between  the  programs  being  run  by  suceessful 
eompanies  and  DoD  aequisition  programs.  The  following  topics  were  discussed  as  major 
differences  between  commercial  companies  studied  by  the  GAO  and  the  DoD  aequisition 
proeess. 

1.  Funding 

One  of  the  key  issues  noted  in  the  GAO  report  was  the  competition  for  funding. 
While  the  DoD  is  very  good  at  developing  long-term  strategie  plans,  those  plans  do  not 
appear  to  lead  to  a  long-term  realistic  investment  strategy  including  weapon 
development.  Instead,  there  is  severe  competition  beeause  more  programs  are  started  than 
ean  be  funded,  this  in  turn,  leads  to  inereased  competition  and  overly  optimistie  estimates 
regarding  cost,  schedule  and  performanee  of  the  competing  programs.  The  GAO  has 
continually  identified  problems  with  the  DoD  budget  proeess  including  this  assessment  in 
2003: 


Sinee  the  mid-1980s,  we  have  reported  that  the  DoD  employs  overly 
optimistic  planning  assumptions  in  its  budget  formulation.  As  a  result, 

DoD  has  too  many  programs  for  the  available  dollars,  whieh  often  leads  to 
program  instability,  eostly  program  stretchouts,  and  program  termination. 

In  2000,  we  reported  that  beeause  the  fiscal  year  2001  program's  projeeted 
cost  was  $16  billion  more  than  the  cost  projected  for  the  same  elements  in 
fiscal  year  2000  program,  DoD  eould  not  implement  its  operation  and 
maintenanee  and  procurement  programs  as  planned.  Over  the  past  few 
years,  the  mismatch  between  programs  and  budgets  has  eontinued, 
espeeially  in  the  area  of  weapon  systems  aequisition. 

The  problems  identified  by  the  GAO  have  also  been  identified  by  DoD  program 
managers.  These  comments  were  taken  direetly  from  program  managers  interviewed  by 
the  GAO: 


Government  Aeeountability  Offiee,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  20. 

GAO,  Major  Management  Challenges  and  Program  Risks,  GAO-03-98,  2003,  11. 
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•  OSD  staff  has  reduced  funding  without  any  understanding  or 
appreciation  for  program  impacts.  It  appears  that  the  staff 
makes  arbitrary  cuts. 

•  OSD  has  a  very  near-term  execution  year  focus,  resulting  in 
great  instability.  In  reality,  it  should  provide  much  more 
strategic  vectors  for  the  Department  instead  of  short-term 
adjustments  to  fix  more  tactical-level  funding  needs. 

•  My  experience  is  that  the  service  and  OSD  typically  cut 
programs  to  pay  top  down  bills. 

•  There  is  no  such  thing  as  funding  stability  in  DoD.  Funding 
reductions  and  programs  stretchouts  are  the  norm  due  to  top 
down  fiscal  bills  that  occur  during  the  execution  year.  The 
pentagon  must  pay  the  bills,  therefore  it  takes  funds  from  the 
programs,  thereby  contributing  to  program  stretchout,  cost 
increases,  inefficiencies,  etc. 

•  Unstable  funding  results  in  pressure  to  do  aggressive  things  in 
order  to  minimize  the  impact  of  budget  cuts  on  schedule  and 
performance.  I  believe  this  has  been  a  major  factor  in 
recent. .  .program  execution  problems. 

•  Our  product  is  considered  a  support  function.  When  funding 
gets  tight,  we  have  been  considered  a  bill  payer  for  others,  even 
if  it  has  "broken"  our  program.  20 

The  next  chart  depicts  the  results  when  the  GAO  asked  DoD  program  managers 
about  the  obstacles  they  faced. 


20  Government  Accountability  Office,  Best  Pmctices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  40. 
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36 


Figure  5.  How  Program  Managers  Responded  to  an  Open-ended  Question  on  What 
Were  the  Biggest  Obstacles  They  Faced  (From;  GAO,  GAO-06-110,  2006) 

In  contrast,  key  leaders  within  the  companies  the  GAO  researched,  ensured  that 
the  projects  undertaken  supported  the  investment  strategy  of  the  company.  This  process 
gives  the  assigned  program  manager  confidence  that  the  leadership  is  committed  to  the 
program  and  there  is  a  clear  understanding  of  funding  priorities.  It  is  interesting  to  note 
that  the  GAO  found  in  their  research  that  not  one  program  manager,  from  any  of  the 
companies,  ever  mentioned  funding  as  a  problem  during  the  program,  funding  was  a 
given  once  senior  leaders  had  committed  to  their  project.21 

2,  Requirements 

The  companies  the  GAO  visited  achieved  there  overall  investment  strategy  by 
matching  requirements  to  the  available  resources.  All  the  key  resources  needed,  including 

Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  25. 
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time,  money,  technology  and  people,  were  available  before  the  program  began.  The 
companies  had  already  conducted  extensive  research  and  all  requirements  were  clearly 
defined  prior  to  initiation  and  commitment  to  the  program.  Previous  research  conducted 
by  the  GAO  in  the  best  practices  series  had  consistently  found  that  matching 
requirements  and  resources  prior  to  initiating  a  new  program  to  be  a  hallmark  for 
successful  companies.  While  research  on  DoD  programs  noted  that  requirements  were 
often  not  fully  defined  at  the  onset  of  a  program,  and  many  also  pointed  out  that  users  and 
stakeholders  often  did  not  adhere  to  the  agreements  made  when  programs  were  launched, 
especially  if  technologies  did  not  mature  as  planned.22 

3.  Staffing 

The  GAO  report  06-110  identified  staffing  issues  as  an  important  concern  of  both 
program  managers  and  program  executive  officers  within  the  DoD.  They  pointed  to 
critical  shortages  of  staff  in  the  areas  of  program  management,  systems  engineering,  cost 
estimating  and  software  development.23  The  GAO  also  identified  practices  within 
successful  companies  that  produced  highly  qualified  program  managers.  The  companies 
placed  a  high  value  on  strong  leadership  qualities  including,  decision  making  skills, 
diplomacy,  communication  skills  and  the  ability  to  motivate  others.  The  program 
managers  interviewed  by  the  GAO  had  a  combination  of  formal  training  and  informal 
mentorship  by  senior  executives.  The  companies  that  were  visited  also  had  formal 
process  for  developing  and  deploying  experts  to  assist  the  program  manager.  The 
program  managers  had  a  high  degree  of  confidence  in  their  support  staff. 

C.  DELANO’S  RESEARCH 

Further  research  was  published  in  the  winter  1998  issue  of  Acquisition  Review 
Quarterly  (now  Acquisition  Review  Journal)  that  supported  the  idea  of  Critical  Success 
Factors.  Major  Kenneth  Delano's  research  concluded  that  there  were  factors  in  DoD 
acquisition  programs  that  could  be  identified  as  being  key  to  the  success  of  programs. 
Delano's  research  was  conducted  using  survey  questions  and  a  literature  review.  Of  the 
32  surveys  sent  to  PMs,  18  were  returned.  This  resulted  in  a  56  percent  rate  of  return 

22  Government  Aeeountability  Offiee,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  42. 

23  Ibid,  44-45. 
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which  was  sufficient  to  validate  the  survey  results.  The  next  table  depiets  the  results  of 
the  survey.  The  highlighted  areas  within  the  tables  will  be  analyzed  further  in  Chapter  4. 


Average  Score 

Program  Success  Factor 

4.42 

Program  Manager's  ability  to  eommunieate 

4.25 

Type  and  quality  of  people  associated  with  the  program 

4.25 

Program  managers  ability  to  lead 

4.25 

Good  relationship  with  user  organization 

4.17 

Resources:  people,  facilities,  money 

4.08 

Product  requirements  and  design  stability 

3.91 

Funding  stability 

3.83 

Good  relationship  with  prime  eontraetor 

3.58 

Program's  aequisition  strategy 

3.58 

Program  manager's  aequisition  experienee 

3.25 

Program  personnel  eontinuity 

3.00 

Program  manager  eontinuity 

3.00 

High  degree  of  technical  difficulty 

2.92 

Program  managers  field  experienee 

2.67 

Program  managers  teehnieal  ability 

2.33 

Total  quality  management  program 

2.25 

Low  degree  of  teehnieal  diffieulty 

5  =  Critical  Factor 

3  =  Important  Factor  1  =  Not  Very  Important  Faetor 

4  =  Very  Important  Faetor  2  =  Somewhat  Important  Faetor 

Table  4.  Delano's  Program  Sueeess  Faetors  Rank  by  Importanee  (From;  Identifying 

Faetors  That  Contribute  to  Program  Sueeess,  1998) 


Delano's  literature  review  eonsisted  of  researehing  19  artieles,  doeuments  and 
books.  He  then  eategorized  key  faetors  into  two  groups;  Aequisition  Faetors  and 
Resouree  Faetors.  These  eategories  were  then  measured  as  to  the  number  of  times  the 
faetor  was  mentioned  in  an  artiele  and  he  eoneluded  through  his  ealeulations,  that  faetors 
with  a  correlation  higher  than  32  pereent  were  considered  the  most  signilioant.24  While 
the  faetors  that  Delano  used  do  not  exactly  mirror  the  factors  Dobbins  used  there  are 
many  eorrelations  and  enough  data  to  eompare  the  two.  The  results  of  Delano's  literature 
review  are  shown  in  the  following  chart. 


24  Kenneth  J.  Delano,  Identifying  Factors  That  Contribute  to  Program  Success,  Acquisition  Review 
Quarterly  Winter- 1998,  1998,  42. 
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Factor 


Source 


Total 


% 


A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

M 

N 

O 

p 

Q 

R 

s 

Well  defined 
requirements 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

47 

Aequisition 

strategy 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

47 

Works  well 
when  fielded 

X 

X 

X 

X 

X 

X 

6 

32 

Stability 

X 

X 

X 

X 

X 

X 

6 

32 

Good 

relations 

with 

eontraetor 

X 

X 

X 

X 

4 

21 

TQM 

program 

X 

X 

X 

3 

16 

Meets 

performanee 

objeetives 

X 

X 

2 

11 

Meets  eost 
objeetive 

X 

1 

5 

Meets  IOC 
date 

0 

0 

Table  5.  Delano's  Aequisition  Factor  Analysis  (From:  Identifying  Factors  That 

Contribute  to  Program  Success,  1998) 


Factor 

Source 

Total 

% 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

M 

N 

o 

p 

Q 

R 

s 

Quality 

people 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

47 

PM 

responsibility 
&  authority 

X 

X 

X 

X 

X 

X 

X 

X 

8 

42 

Total  team 
eoneept 

X 

X 

X 

X 

X 

X 

X 

7 

37 

PM  skills 

X 

X 

X 

X 

X 

X 

X 

7 

37 

Congressional 

involvement 

X 

X 

X 

3 

16 

User 

involvement 

X 

X 

2 

10 

Adequate 

resourees 

X 

X 

2 

10 

Adequate 

staffing 

X 

1 

5 

25 


Factor 

Source 

Total 

% 

Spt  agency 
involvement 

X 

1 

5 

Higher  Cmd 
involvement 

X 

1 

5 

PM  technical 
ability 

X 

1 

5 

GAO 

involvement 

0 

0 

Table  6.  De 


ano's  Resource  Factor  Analysis  (From;  Identifying  Factors  That 
Contribute  to  Program  Success,  1998) 


A 

Baumgartner,  Brown  and  Kelly 

B 

Beltramo 

C 

Clay 

D 

Gansler 

E 

Gregory 

F 

Heberling  and  Graham 

G 

Hicks,  Rich,  Wertheim  and  Meyer 

H 

Hirsch  and  Waelchli 

I 

Kish 

J 

Lesser 

K 

Nelson 

L 

Price  and  Valentine 

M 

"RX  for  Ailing  Procurement  System" 

N 

Sammet  and  Green 

0 

Settlmeyer 

P 

Snoderly  and  Acker 

0 

Total  Quality  Management  Master  Plan 

R 

Weiss 

S 

Zairi 

Table  7.  Correspondence  Between  Codes  and  Sources  (From;  Identifying  Factors  That 


Contribute  to  Program  Success,  1998) 

Delano's  research  concluded  that  the  following  factors  were  ranked  high  both  in  the 
literature  review  and  the  PM  survey  and  considered  key  to  the  success  of  DoD  acquisition 
programs. 


•  Well  defined  requirements 

•  Acquisition  strategy 

•  Actual  weapon  system  performance 
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•  Program  stability 

•  Quality  people 

•  PM  authority 

•  Total  team  eoneept 

•  PM  skills25 

While  the  data  that  is  presented  in  Delano's  paper  has  distinet  differenees  there  are 
many  similarities  to  the  researeh  done  by  Dobbins  and  Donnelly.  Some  of  the  differenees 
in  the  survey  results  eould  be  explained  by  the  questionnaire  format  and  foeus  of  the 
surveys,  but  the  main  point  of  Delano's  researeh  is  the  same  as  Dobbins'  and  Donnelly's, 
that  there  are  faetors  eommon  to  most  programs  that  if  they  are  managed  and  eontrolled 
will  lead  to  a  sueeessful  program. 

D.  RAND  STUDY 

In  1996,  Robert  Johnson  and  John  Birkler  published  "Three  Programs  and  Ten 
Criteria"  whieh  was  sponsored  by  the  Offiee  of  the  Seeretary  of  Defense  (OSD)  and 
addressed  issues  within  DoD  aequisition  management.  The  authors  used  their 
observations  eombined  with  lessons  learned  from  past  programs,  as  well  as  their 
knowledge  of  DoD  aequisition  management  to  develop  a  struetured  list  of  "key  faetors" 
to  be  eonsidered  in  sueeessfully  managing  a  major  defense  aequisition  program.  The 
following  is  a  list  of  faetors  that  Johnson  and  Birkler  identified  as  eritieal  to  the  sueeess 
of  a  weapons  system  program: 

•  Lines  of  authority  have  been  established  and  are  elear.  Defense 
Management  Review  issues  and/or  problems  must  not  eause 
eonfusion,  biekering,  or  a  diminution  of  Program  Manager 
responsibility  and  aeeountability. 

•  Communieation  is  open  (no  seerets-all  information  is  divulged; 
using  all  media  and  avenues,  e.g.,  email,  written,  verbal)  and 
eontinuous  at  and  between  all  levels  of  authority. 


Kenneth  J.  Delano,  Identifying  Factors  That  Contribute  to  Program  Success,  Acquisition  Review 
Quarterly  Winter- 1998,  1998,43. 


27 


•  Cost/Schedule  Control  System  (CS2),  cost  performance 
measurement  (CPM),  and  other  management  reports  are  used  as 
indicators  of  trends  in  program  progress  and  for  reporting  program 
status. 

•  Risk-management  techniques  have  been  implemented. 

•  Program  stability  has  been  achieved  through  control  of 
requirements. 

•  A  strong  Government-industry  support  team  (Program  Office, 
functional  support,  Defense  Plant  Representative  Offices)  is 
present  and  has  explicit  mechanisms  for  coordinating 
responsibilities. 

•  Incentives  for  the  Program  Manager  are  adequate  and  positive. 

•  Funding  is  stable  and  adequate. 

•  Selection  of  best-qualified  personnel  for  key  acquisition 
management  positions  is  objective  and  regulated. 

•  Security  requirements  do  not  restrict  adequate  and  sufficient 
management.26 

The  author  used  this  list  of  factors  to  form  a  baseline  for  a  well  managed  program  and  see 

the  above  factors  as  critical  to  managing  a  successful  acquisition  program.  The  author's 

main  points  listed  above  focus  on  the  same  areas  as  research  conducted  by  Dobbins  and 

Donnelly  as  well  as  Delano  and  find  that  areas  such  as  requirements,  funding,  risk 

management  and  staffing  are  critical  to  the  success  of  DoD  acquisition  programs. 

E.  LESSONS  LEARNED  FROM  THE  F/A-22  AND  F/A-18E/F 
DEVELOPMENT  PROGRAMS 

At  the  behest  of  the  Air  Force  the  Rand  Corporation  conducted  a  review  of  the 
F/A-22  and  F/A-18E/F  programs  in  order  to  identify  lessons  learned  that  could  be  applied 
to  future  programs.  The  results  were  published  in  2005  and  provide  useful  information  to 


Robert  V.  Johnson  and  John  Birkler,  Three  Programs  and  Ten  Criteria-Evaluating  and  Improving 
Acquisition  Program  Management  and  Oversight  Processes  Within  the  DoD,  1996  Rand,  7-8. 
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future  program  managers  regarding  eritieal  faetors  and  their  impaet  on  the  outcome  of 
DoD  acquisition  program.  The  Rand  study  identified  the  following  lessons  learned: 


•  Early,  realistic  cost  and  schedule  estimates  set  the  program  on  the 
right  path  for  the  rest  of  the  development  program. 

•  A  stable  development  team  structure,  proper  team  expertise,  clear 
lines  of  responsibility  and  authority  and  a  lead  contractor 
responsible  for  overall  program  progress  are  critical  to  program 
success. 

•  An  experienced  management  team  and  contractors  with  prior 
business  relationships  help  eliminate  early  management  problems. 

•  Concurrent  development  of  new  technology  for  the  airframe, 
avionics  and  propulsion  adds  significant  risk. 

•  Reducing  the  cost  and  risk  of  avionics  should  be  a  key  focus  of  the 
concept  development  phase.  Avionics  is  a  considerable  cost  driver 
of  modem  weapon  systems,  and  new  concepts  should  be 
demonstrated  along  with  new  airframe  designs. 

•  Preplanned,  evolutionary  modernization  of  high-risk  avionics  can 
reduce  risk  and  help  control  cost  and  schedule. 

•  Careful  monitoring  of  airframe  weight  is  important.  Airframe 
weight  instability  is  an  early  indicator  of  problems. 

•  Earned  Value  Management  (EVM)  data  should  be  used  to  monitor 
and  manage  program  costs  at  the  level  of  integrated  product  teams 
(IPTs).  27 


F.  GAO  REPORTS 

Several  GAO  reports  are  also  relevant  to  the  research  on  Critical  Success  Eactors 
within  DoD  Acquisition  programs.  While  the  data  provided  in  these  reports  does  not 


27  Obaid  Younossi ,  David  E.  Stem,  Mark  Lorell,  Frances  M.  Fussier,  Lessons  Learned  from  F/A-22 
and  F/A-18  E/F  Development  Programs,  2005,  Rand,  57. 


29 


specifically  address  Critical  Success  Factors  these  GAO  reports  present  separate  issues 
that  programs  have  encountered  during  their  development  that  are  closely  tied  to  Critical 
Success  Factor  Theory. 

1.  GAO-06-585T  Defense  Acquisitions-Actions  Needed  to  Get  Better 

Results  on  Weapon  Systems  Investments 

This  document  outlines  the  major  issues  with  DoD  acquisitions  from  the  GAO's 
perspective  and  then  provides  solutions  that  it  believes  will  address  the  problems  that  the 
DoD  faces.  Several  of  the  problems  they  identify  and  solutions  they  suggest  are  similar  to 
those  that  are  addressed  by  the  Critical  Success  Factors.  The  first  issue,  is  the  problem  of 
well  defined  and  stable  requirements.  Regarding  requirements,  the  GAO  stated: 

...DoD  has  exacerbated  their  problems  by  not  clearly  defining  and 
stabilizing  requirements  before  programs  are  started.  At  times,  in  fact,  it 
has  allowed  new  requirements  to  be  added  well  into  the  acquisition  cycle- 
significantly  stretching  technology  and  creating  design  challenges,  and 
exacerbating  budget  overruns.  For  example,  in  the  F-22A  program,  the  Air 
Force  added  a  requirement  for  air-to-ground  attack  capability.  In  its 
Global  Hawk  program,  the  Air  Force  added  both  signal  intelligence  and 
imagery  intelligence  requirements. 

In  order  to  address  the  problem  the  GAO  recommends  that  system  requirements  are 
agreed  by  the  service  acquisition  executives  and  the  warfighters  and  that  no  additional 
requirements  are  added  during  execution  unless  they  are  fully  funded. 

Another  issue  that  the  GAO  addresses  is  funding  of  weapons  systems.  The  GAO 
report  states: 

DoD  starts  more  weapons  programs  than  it  can  afford  and  sustain,  creating 
a  competition  for  funding  that  encourages  low  cost  estimating,  optimistic 
scheduling,  over  promising  and  suppressing  of  bad  news.  Programs  focus 
on  advocacy  at  the  expense  of  realism  and  sound  judgment.  Invariably, 
with  too  many  programs  in  its  portfolio,  DoD  and  the  Congress  are  forced 
to  continually  shift  funds  to  and  from  programs.29 


Government  Accountability  Office,  Defense  Acquisition- Actions  Needed  to  get  Better  Results  on 
Weapons  Systems  Investments,  GAO-06-585T,  2006,  6. 

Government  Accountability  Office,  Defense  Acquisition-Actions  Needed  to  get  Better  Results  on 
Weapons  Systems  Investments,  GAO-06-585T,  2006,  6. 
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Their  solution  is  to  enforce  funding  for  priorities  annually  and  measure  success 
against  the  original  plan.  The  GAO  report  addresses  many  other  issues  outside 
the  scope  of  this  research  which  is  not  pertinent  to  the  discussion  on  Critical 
Success  Factors. 

2.  GAO-03-98  Major  Management  Challenges  and  Program  Risks 

This  report  was  written  as  part  of  a  series  on  performance  and  accountability  and 
identifies  systemic  and  specific  problems  with  management  processes.  While  this  report 
does  not  address  Critical  Success  Factors  directly,  some  of  the  systemic  issues  the  GAO 
found  within  the  DoD  have  implications  with  regard  to  CSFs.  Specifically,  The  GAO 
findings  regarding  technical  risk  and  critical  technology  maturity.  The  GAO  report  stated, 
"...we  have  found  major  weapon  systems  at  risk  of  not  being  able  to  meet  program 
objectives  because  critical  technologies  were  immature  and  software  development  was 
not  effectively  managed,  The  systems  that  are  at  risk  because  of  immature  technology 
include  the  Joint  Strike  Fighter,  the  Airborne  Laser  and  the  Space-based  Infrared  System. 
The  report  focuses  on  several  management  challenges  but  the  main  point  regarding 
Critical  Success  Factors  is  that  risk  management  and  specifically  technological  risk  is  a 
major  concern  for  weapon  system  programs. 

3.  GAO-04-393  Defense  Acquisitions-Stronger  Management  Practice  are 
Needed  to  Improve  DoD’s  Software-Intensive  Weapon  Acquisitions 

This  GAO  report  focused  on  problems  related  to  DoD's  management  of  software 

and  software  development  within  weapon  systems.  The  GAO  visited  three  leading 

commercial  software  developers  to  identify  procedures  that  could  improve  DoD's 

software  development  processes.  While  many  issues  are  addressed  in  this  report,  the 

section  regarding  requirements  is  relevant  to  the  discussion  of  Critical  Success  Factors. 

The  GAO  found: 

Senior  managers  at  software  development  and  acquisition  companies  we 
visited  expect  requirements  to  be  managed  and  controlled  before  design 
work  begins  and  virtually  all  lower-level  design  elements  to  be  adequately 
defined  before  the  start  of  coding.  Without  adequate  definition  and 
validation  of  requirements  and  design,  software  engineers  could  be  coding 

Government  Accountability  Office,  Major  Management  Challenges  and  Program  Risks,  GAO-03- 
98,  2003,  54. 
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to  an  incorrect  design,  resulting  in  missing  funetionality  or  errors. 
Motorola  CSG,  a  communieations  company,  and  Teradata,  a  division  of 
NCR  that  specializes  in  database  teehnology,  estimate  that  95  pereent  of 
their  requirements  are  set  by  the  end  of  the  requirements  phase  and  98 
pereent  by  the  end  of  the  design  phase.  Officials  view  managing 
requirements  as  the  most  eritieal  development  task  to  ensure  sueeessful 
software  outcomes.^i 

The  report  found  that  stable  and  defined  requirements  are  eritieal  to  the  success  of  any 

software  development  proeess.  As  weapon  systems  beeome  more  complex  and  dependent 

on  software  the  ability  to  stabilize  and  clearly  define  requirements  becomes  even  more 

erucial  for  the  sueeessful  aequisition  of  weapons. 

4,  GAO-03-55  Acquisition  Workforce  -  Status  of  Agency  Efforts  to 

Address  Future  Needs 

This  report  foeuses  on  problems  regarding  the  size  and  quality  of  the  workforce 
within  Government  organizations.  The  GAO  identifies  the  work  the  DoD  has  done  on 
human  capitol  strategy  as  a  model  for  other  organizations  to  follow.  The  GAO 
acknowledges  that  the  DoD  has  room  for  improvement  but  is  farther  along  than  most 
organizations  and  stated  the  following,  "DOD  has  been  working  for  several  years  to 
strengthen  its  civilian  acquisition  workforce.  The  acquisition  workforce  comprises  a  large 
proportion  of  the  overall  workforce,  and  DOD  views  the  acquisition  workforce  as  critical  to 
accomplishing  its  mission. "32 

Improving  the  quality  of  the  DoD  workforce  has  been  part  of  the  strategy  within  the  DoD 
for  many  years  and  incorporates  guidance  from  the  Defense  Acquisition  Workforce  Improvement 
Act  (DAWIA),  the  Clinger-Cohen  Act  and  other  policies  to  ensure  the  acquisition  workforce  has 
the  necessary  skills  that  are  required  to  assist  in  the  successfiil  acquisition  of  weapon  systems. 
While  the  DoD  has  identified  the  need  for  a  professional  workforce  the  downsizing  of  the  same 
workforce  has  caused  several  problems,  including  the  following  reported  by  the  GAO: 

During  the  past  decade,  DOD  has  downsized  its  civilian  acquisition 
workforce  by  half.  It  now  faces  what  it  considers  to  be  serious  imbalances 
in  the  skills  and  experience  of  its  remaining  workforce  and  the  potential 
loss  of  highly  specialized  knowledge  if  many  of  its  acquisition  specialists 

31  Government  Accountability  Office,  Defense  Acquisitions-Stronger  Management  Practices  are 
Needed  to  Improve  DoD's  Software-Intensive  Weapon  Acquisitions,  GAO-04-393,  2004,  14. 

32  Government  Accountability  Office,  Acquisition  Workforce-Status  of  Agency  Effort  to  Address 
Future  Need,  GAO-03-55,  2003,  16. 
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retire.  DOD  ereated  the  Aequisition  2005  Task  Foree  to  study  this  problem 
and  develop  a  strategy  to  replenish  personnel  losses.  The  task  foree’s  first 
reeommendation  was  to  develop  and  implement  a  human  eapital  strategie 
plan  for  the  eivilian  aequisition  workforoe.33 

The  report  identified  the  need  to  have  a  well-trained,  professional  and  eertified  workforce 
that  has  the  right  mix  of  skills  and  is  sustainable  in  order  to  assist  their  organizations  to 
be  effective. 

G.  CONCLUSION 

This  chapter  presented  data  related  to  Critical  Success  Factors  from  a  variety  of 
sources  including  the  Government  Accountability  Office,  Rand  Corporation  and  several 
authors.  The  next  chapter  will  illustrate  that  Critical  Success  Factors  are  relevant  to  DoD 
acquisition  programs  and  can  be  used  to  successfully  manage  programs.  Furthermore,  it 
will  show  the  relationship  between  the  variety  of  research  that  has  been  done  by  the 
different  authors  and  organizations. 


33Govemment  Accountability  Office,  Acquisition  Workforce-Status  of  Agency  Effort  to  Address 
Future  Need,  GAO-03-55,  2003,  17. 
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IV.  ANALYSIS  OF  CRITICAL  SUCCESS  FACTORS 


There  is  signifieant  data  regarding  the  failure  of  DoD  acquisition  programs  and 
what  steps  the  DoD  and  services  should  take  to  "fix"  the  problems  with  the  acquisition 
system.  For  example,  the  GAO  has  complete  series  of  reports  dedicated  to  improving 
DoD  acquisition,  such  as  the  Best  Practices  Series  which  identifies  industry  leaders  and 
applies  there  best  practices  to  government  applications.  The  data  in  chapter  three  is  an 
attempt  by  professionals  and  government  organizations  to  identify  issues  that  lead  to  the 
failure  of  a  program  and  identify  corrective  actions  in  order  to  create  a  more  favorable 
outcome  with  future  DoD  programs.  The  data  gathered  comes  from  many  sources 
including,  lessons  learned  from  past  programs,  this  includes  reports  such  as  Three 
Programs  and  Ten  Criteria^^,  interviews  with  program  managers,  such  as  those  conducted 
by  the  GAO  for  the  report  GAO-06-1 10^5,  interviews  and  studies  of  program  managers 
and  programs  in  the  civilian  sector  as  well  as  professional  experience,  such  as  the  Rand 
report  Lessons  Learned  from  the  F/A-22  and  F/A-18  E/F  Development  Programs.36  The 
following  factors  have  been  identified  by  several  authors  or  publications  as  critical  to  the 
success  of  programs  and  support  Dobbins  conclusions  that  there  are  Critical  Success 
Factors  common  to  most,  if  not  all  DoD  acquisition  programs.  Dobbins'  and  Donnelly's 
research  found  18  CSFs  that  they  considered  critical  to  successful  programs  and  relevant 
to  all  DoD  programs.  While  most  other  data  does  not  find  the  exact  results  that  Dobbins 
and  Donnelly  did,  other  research  has  found  elements  with  key  similarities.  For  example, 
Dobbins  and  Donnelly  use  the  category  of  "Risk  Management"  for  one  of  their  Critical 
Success  Factors,  while  Delano  divides  risk  management  into  several  categories  including 
"High  degree  of  technical  difficulty"  and  "Program's  acquisition  strategy".  These 
categories  are  not  identical  but  are  similar  when  they  are  applied  to  acquisition  programs. 


Robert  V.  Johnson  and  John  Birkler,  Three  Programs  and  Ten  Criteria-Evaluating  and  Improving 
Acquisition  Program  Management  and  Oversight  Processes  Within  the  DoD,  1996  Rand. 

Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006. 

3^  Obaid  Younossi ,  David  E.  Stem,  Mark  Lorell,  Frances  M.  Fussier,  Lessons  Learned  from  F/A-22 
and  F/A-18  E/F  Development  Programs,  2005,  Rand. 
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A.  STABLE  AND  ADEQUATE  FUNDING 

In  Dobbins  and  Donnelly's  research  among  DoD  Program  Managers  stable  and 
adequate  funding  is  the  number  one  Critical  Success  Factor  identified  by  both  the 
Embedded  System  PMs  and  the  Automated  Information  Systems  PMs.^^  The  GAO  found 
the  same  results  when  interviewing  PMs,  and  Figure  4  clearly  illustrates  that  PMs 
identify  funding  instability  as  the  number  one  issue  when  addressing  DoD  acquisition 
management.38  Delano's  research  as  well  as  Johnson  and  Birkler's  also  found  that  stable 
funding  was  critical  to  the  success  of  acquisition  programs.  With  regard  to  funding 
stability  the  Johnson  and  Birkler  stated,  "Budget  instability  plagues  all  three  programs 
(F/A-18EF,  F-22,  RAH-66)  and  causes  the  greatest  concern  for  acquisition-management 
officials.  Failure  to  meet  this  criterion  was  the  most  seriously  detrimental  aspect  we 
found  during  our  research  on  the  three  programs. "39  The  GAO  found  that  among  PMs  in 
the  private  sector  stable  funding  was  not  a  serious  concern  during  the  life  of  a  program. 
The  private  sector  PM  must  worry  about  the  cost  of  his  program  and  ensure  that  the  cost 
is  within  a  narrow  band  of  the  cost  estimates  but  once  a  strategic  decision  is  made  for  a 
new  product,  the  PM  has  the  support  and  understands  that  the  senior  leaders  are  fully 
committed  to  the  program. 

B,  CLEARLY  DEFINED  AND  STABLE  REQUIREMENTS 

As  presented  in  the  previous  chapter,  Dobbins'  and  Donnelly's  research  found  that 
stable  requirements  was  ranked  second  among  CSFs  and  was  mentioned  17  times  in  the 
returned  surveys.^o  Their  research  found  that  clear  and  stable  requirements  were  critical 
to  the  success  of  acquisition  programs. 

Further  research  by  the  GAO  and  others  has  also  shown  that  requirements  are 
critical  to  the  success  of  programs.  The  GAO  report  06-110  found  that  DoD  program 


37  James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  69-70. 

3^  Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  44. 

39  Robert  V.  Johnson  and  John  Birkler,  Three  Programs  and  Ten  Criteria-Evaluating  and  Improving 
Acquisition  Program  Management  and  Oversight  Processes  Within  the  DoD,  1996  Rand,  55. 

James  H.  Dobbins  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  71. 
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managers  saw  requirements  instability  as  a  major  obstaele  to  their  program.  Program 
managers  interviewed  by  the  GAO  made  the  following  eomments:  "...that  requirements 
were  often  not  fully  defined  at  the  onset  of  the  program  and  stakeholders  often  did  not 
stiek  to  the  agreements  they  made  when  programs  were  launehed,  especially  if 
technologies  did  not  mature  as  planned."^!  When  DoD  program  managers  were  asked  to 
respond  to  the  open-ended  question  regarding  the  biggest  obstacle  to  their  program, 
requirements  instability  was  mentioned  often  and  only  less  than  funding  instability  as  a 
major  concern.42  The  Report  also  showed  that,  in  companies  that  the  GAO  reviewed, 
requirements  were  clearly  identified  and  the  senior  leadership  ensured  that  the  company 
had  the  resources  to  meet  the  requirements  set  out  in  the  program.  The  companies  will 
only  fund  programs  with  clearly  defined  and  stable  requirements.  While  the  GAO 
reported  the  following  regarding  DoD  programs,  "...program  managers  commented  that 
requirements  continue  to  be  added  as  the  program  progresses  and  funding  instability 
continues  throughout.  These  two  factors  alone  cause  the  greatest  disruption  to  programs, 
according  to  program  managers. "^3 

Major  Delano's  research  also  concluded  that  product  requirements  and  design 
stability  was  considered  very  important  among  the  respondents  to  his  survey.  As  the 
tables  in  the  previous  chapter  showed.  Product  Requirements  and  Design  Stability  (4.08 
out  of  5.00)  was  considered  very  important  in  the  success  of  a  program.  Delano's 
literature  review  also  had  requirements  ranked  high  among  the  books,  articles  and 
journals  he  examined.  In  his  analysis  of  acquisition  factors  Delano  found  that  "well 
defined  requirements"  was  mentioned  47  percent  of  the  time.  He  found  this  to  be 
significant  and  determined  that  it  is  critical  to  the  success  of  a  program.  From  the  data 
presented  in  the  previous  chapter,  there  appears  to  be  a  correlation  between  successful 
weapon  systems  programs  and  clearly  defined  and  stable  requirements. 

While  it  seems  simple  to  say  that  DoD  programs  need  clearly  defined  and  stable 
requirements,  in  practice,  it  is  difficult  to  achieve.  DoD  programs  often  experience 

Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  42. 

42  Ibid,  44. 

43  Ibid,  45. 
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changes  to  requirements  due  to  several  faetors  ineluding;  length  of  the  program, 
emerging  teehnologies,  ehanges  in  mission  or  changes  in  funding.  Frequently  the  need  for 
a  weapon  no  longer  exists  but  the  serviees  do  not  want  to  cancel  the  program  beeause 
extensive  time  and  money  has  gone  into  the  program,  so  instead  of  treating  the  program 
as  a  sunk  eost  and  cancelling  it  the  services  develop  a  weapon  system  that  is  not  needed. 
C.  RISK  MANAGEMENT 

Dobbins'  and  Donnelly's  researeh  concluded  that  risk  management  was  a  Critical 
Success  Faetor  within  DoD  aequisition  programs.  Risk  Management  eneompasses  all 
aspects  of  risk  within  a  program  to  include  teehnological  risk  as  well  as  funding  and 
sehedule  risk.  In  2002  Dobbins  deseribed  the  process  of  risk  management; 

The  fundamental  notion  is  that  we  identify  risk,  we  assess  their  probability 
of  oecurrence,  and  we  assess  the  consequence  of  occurrence.  Then  we  put 
a  risk  management  plan  in  plaee  that  is  designed  to  alleviate  the  impact  of 
the  serious  event.  Every  risk  is  neeessarily  a  future  event,  and  only  when 
the  risk  event  actually  happens  is  the  risk  transformed  into  a  problem.  The 
better  we  are  at  identifying  risks  and  understanding  the  underlying  basis  of 
our  risks,  the  better  we  ean  manage  the  risks. 44 

The  DoD  Guide  to  Risk  Management  defines  risk  and  risk  management  in  the  following 
manner,  "Risk  is  a  measure  of  the  potential  inability  to  aehieve  overall  program 
objectives  within  defined  eost,  sehedule,  and  teehnieal  constraints  and  has  two 
components:  (1)  the  probability/likelihood  of  failing  to  achieve  a  partieular  outeome,  and 
(2)  the  consequences/impacts  of  failing  to  achieve  that  outcome."  It  goes  on  to  define  the 
process  of  risk  management  as: 

Risk  management  is  the  aet  or  practice  of  dealing  with  risk.  It  includes 
planning  for  risk,  assessing  (identifying  and  analyzing)  risk  areas, 
developing  risk-handling  options,  monitoring  risks  to  determine  how  risks 
have  changed,  and  doeumenting  the  overall  risk  management  program.^^ 


James  Dobbins,  Critical  Success  Factor  Analysis  for  DoD  Risk  Management,  Program  Management 
Journal,  May  2002,  40. 

Risk  Management  Guide  for  DoD  Aequisitions  (5th  ed.),  2003,  7. 
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Dobbins  and  Donnelly  describe  risk  management  in  broad  terms  and  use  the 
overarching  concept  of  risk  management  as  one  of  their  Critical  Success  Factors.  As 
Dobbins  and  Donnelly  use  the  term  "risk  management",  all  aspects  of  a  program  can  be 
considered  part  of  the  risk  management  process,  from  funding  stability  which  causes 
budget  risk  to  technical  immaturity  which  is  one  aspect  of  technical  risk.  Other  research 
has  also  identified  risk  management  as  critical  to  the  success  of  a  program  but  many 
researchers  have  defined  the  elements  of  risk  management.^®  For  example,  Delano  uses 
several  categories  to  describe  risk  including  "Program's  acquisition  strategy",  and  "High 
degree  of  technical  difficulty".  While  neither  of  these  categories  mentions  risk 
management,  both  are  strategies  to  reduce  budget,  schedule  and  technical  risk  and  are 
part  of  an  overarching  risk  management  plan.  It  is  clear  that  Delano  also  sees  risk 
management  as  critical  to  the  success  of  DoD  acquisition  programs. 

The  GAO  has  also  found  issues  with  technical  risk  with  many  of  the  weapon 
systems  being  developed  by  the  DoD.  The  GAO  notes  that  many  weapons  are  developed 
with  immature  technology  that  is  key  to  the  system.  This  usually  creates  unacceptable 
risk  for  most  programs.  When  technologies  are  not  mature  and  the  system  must  have  that 
capability  to  perform  properly,  the  program  runs  the  risk  of  going  over  budget,  or  over 
schedule,  or  both  because  the  key  technology  did  not  mature  fast  enough.  The  GAO 
commented: 

Our  most  recent  annual  assessment  of  major  weapon  systems  programs, 
showed  that  only  15  percent  of  the  programs  we  reviewed  began 
development  having  demonstrated  that  all  of  their  technologies  were 
mature.  More  often  than  not,  programs  had  to  worry  about  maturing 
technologies  well  into  system  development,  when  they  should  have 
focused  on  maturing  system  design  and  preparing  for  production.  These 
assessments  also  show  that  programs  that  started  development  with  mature 
technologies  experienced  lower  development  and  unit  cost  increases  than 
those  programs  that  started  with  immature  technologies.'^^ 


Kenneth  J.  Delano,  Identifying  Factors  That  Contribute  to  Program  Success,  Acquisition  Review 
Quarterly  Winter- 1998,  1998,  39. 

Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  41. 
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As  the  GAO  points  out,  immature  technologies  often  exacerbate  budget  and  schedule 
problems  because  it  takes  longer  and  costs  more  than  if  the  technology  was  already  at  a 
readiness  level  that  was  useful  to  the  program.  This  is  a  theme  that  is  seen  in  repeated  in 
several  GAO  reports,  including  GAO-06-585T,  where  the  GAO  found: 

...DoD  commits  to  its  programs  before  it  obtains  assurance  that  the 
capabilities  it  is  pursuing  can  be  achieved  within  the  available  resources 
and  time  constraints.  Funding  processes  encourage  this  approach,  since 
acquisition  programs  attract  more  dollars  than  efforts  concentrating  solely 
on  proving  out  technologies.  Nevertheless,  when  DoD  chooses  to  extend 
technology  invention  into  acquisition,  programs  experience  technical 
problems  that  have  reverberating  effects  and  require  large  amounts  of  time 
and  money  to 

From  the  data  that  was  presented  in  Chapter  three  it  is  clear  that  risk  management 
is  a  Critical  Success  Factor  and  researchers,  professionals  and  the  GAO  agree  that  risk 
management  must  be  carefully  addressed  in  order  to  develop  successful  weapon  system 
acquisition  programs.  This  is  difficult  to  do  within  the  DoD  because  of  the  need  to 
develop  weapons  with  greater  capability  and  survivability.  Most  program  managers 
would  want  to  include  any  technology  if  they  thought  it  would  save  a  service-members 
life  in  a  combat  environment.  The  trade-off  between  cost  and  performance  is  a  tough 
choice  to  make  when  it  involves  someone  life.  While  trade-offs  must  be  made  it  clear  that 
program  managers  must  manage  risk  in  order  to  develop  a  successful  program. 

D,  TECHNICALLY  COMPETENT  PROGRAM  OFFICE  STAFF 

Dobbins  and  Donnelly  identified  a  technically  competent  program  staff  as  the 
fourth  most  identified  Critical  Success  Factor  mentioned  by  both  Embedded  Systems 
Program  Managers  and  the  Automated  Information  Systems  Program  Managers.  As  was 
pointed  out  in  Chapter  III,  program  managers  specifically  mentioned  expertise  in  systems 
engineering  as  part  of  a  technically  competent  staff.  Further  research  by  others  identified 


Government  Accountability  Office,  Defense  Acquisition-Actions  Needed  to  get  Better  Results  on 
Weapons  Systems  Investments,  GAO-06-585T,  2006,  6. 
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the  same  staffing/personnel  problems,  ineluding  the  program  managers  interviews  and 
literature  review  that  was  conducted  by  Delano,  and  interviews  and  case  studies 
completed  by  the  GAO.  ^9 

Delano's  research  also  identified  a  technically  competent  staff  as  a  key  factor 
within  acquisition  programs.  His  reference  to  "type  and  quality  of  people  associated  with 
the  program"  and  "resources:  people"  are  comparable  to  Dobbins'  and  Donnelly's 
"technically  competent  staff."  Delano's  literature  review  also  found  that  attribute  "quality 
people"  was  considered  critical  and  was  mentioned  in  47  percent  of  the  literature  that  he 
researched. 

Program  managers  interviewed  by  the  GAO,  also  identified  staffing  problems  as  a 
major  obstacle  to  successful  programs.  Specifically,  the  GAO  stated,  "...program 
managers  comments  and  interviews  with  program  executive  officers  pointed  to  critical 
shortages  for  staff  that  support  them-including  program  management,  systems 
engineering,  cost  estimating  and  software  development. 

Further  research  conducted  by  the  GAO  and  the  DoD  have  also  revealed  that 
technically  competent  and  professional  staff  are  a  key  component  of  the  acquisition 
process.  The  GAO  has  done  several  studies  on  the  Government  workforce  and  one  of 
these  reports  was  presented  in  Chapter  Three  as  data  regarding  the  importance  of  a 
technically  competent  workforce  within  the  DoD  and  specifically,  competent  program 
staff  within  DoD  acquisition  programs,  The  DoD  also  has  identified  the  need  to  educate 
and  train  the  acquisition  workforce  to  ensure  they  are  technically  competent  regarding  the 
acquisition  process.  The  Defense  Acquisition  Workforce  Improvement  Act  certification 
was  the  system  that  answered  the  question,  "how  does  the  DoD  ensure  they  have  a 
technically  competent  and  qualified  workforce?"  The  DAU  website  states,  "The  DAU 


^^9  Kenneth  J.  Delano,  Identifying  Factors  That  Contribute  to  Program  Success,  Acquisition  Review 
Quarterly  Winter-1998,  1998,  38. 

Government  Accountability  Office,  Best  Practices-Better  Support  of  Weapon  System  Program 
Managers  Needed  to  Improve  Outcomes,  GAO-06-1 10,  2006,  44-45. 

Government  Accountability  Office,  Acquisition  Workforce-Status  of  Agency  Effort  to  Address 
Future  Need,  GAO-03-55,  2003,  17. 
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provides  a  highly  structured  sequence  of  courses  needed  to  meet  the  mandatory  and 
desired  training  standards  established  in  DoD  5000. 52-M,  “Career  Development  Program 

for  Acquisition  Personnel. ”^2 

The  DoD  continues  to  face  challenges  to  its  workforce.  The  DoD  must  provide 
adequately  trained  and  educated  staff  to  support  the  complex  weapon  system  programs 
that  are  developed  and  produced  by  the  DoD.  Over  the  past  decade  the  DoD  has  reduced 
the  acquisition  workforce  by  half,  while  still  trying  to  maintain  qualified  personnel  to 
assist  program  managers,  the  loss  of  many  experienced  people  has  reduced  the  overall 
effectiveness  of  the  acquisition  system.53  Furthermore,  it  is  imperative  that  acquisition 
professionals  receive  both  education  and  training.  DAWIA  certification  is  a  excellent 
training  tool  but  must  be  coupled  with  education  in  a  specific  discipline  such  as  systems 
engineering,  program  management,  operational  research  etc.  While  the  challenge  to  train 
and  educate  staff  is  critical  it  is  also  critical  to  have  the  right  people  in  sufficient  numbers 
at  the  right  place  to  effectively  staff  a  program  office.  It  is  clear  from  the  data  presented 
in  Chapter  III  that  program  managers  and  other  program  professionals  as  well  as  the 
GAO  have  identified  a  technically  competent  staff  as  a  critical  component  of  a  DoD 
programs. 

E,  SUMMARY 

There  have  been  a  number  of  documents  written  on  Critical  Success  Factors  and 
their  application  to  the  DoD  acquisition  process  and  there  has  also  been  work  on  issues 
that  continue  to  plague  DoD  acquisition  programs.  While  much  of  the  research  does  not 
acknowledge  the  existence  of  Critical  Success  Factors  as  it  applies  to  DoD  acquisition 
many  authors  are  attempting  to  find  a  set  of  criteria  that  can  be  identified  in  most 
programs  that  will  be  predictive  in  nature,  and  allow  program  managers  and  program 
executives  to  use  the  criteria  to  assess  the  probability  of  a  positive  outcome  for  the 
program.  The  analysis  that  has  been  done  in  this  chapter  shows  that  there  is  a  correlation 
between  the  Critical  Success  Factors  researched  and  the  success  of  the  program. 


^2  http://www.dau.mil/plm/plm.asp.  DAU  Website,  2006. 


^3  Government  Aeeountability  Offiee,  Aequisition  Workforee- Status  of  Ageney  Efforts  to  Address 
Future  Needs,  GAO-03-55,  2002,  17. 
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The  research  of  Dobbins  and  Donnelly,  Delano,  The  Rand  Corp  and  The 
Government  Accountability  Office  make  it  clear  that  the  DoD  can  benefit  from  a 
systematic  approach  to  identifying  factors  that  lead  to  the  successful  completion  of 
programs.  While  not  all  programs  will  benefit  from  a  set  of  criteria,  it  is  imperative  that 
Critical  Success  Factors  are  identified  and  measured  for  most  programs  in  order  to  have  a 
repeatable  process  for  predicting  the  success  of  future  DoD  acquisition  programs.  Critical 
Success  Factors  will  help  to  eliminate  some  of  the  variability  in  the  acquisition  process 
and  provide  a  more  stable  acquisition  environment. 
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V.  RECOMMENDATIONS  AND  CONCLUSIONS 


A.  CONCLUSIONS 

The  DoD  is  continuing  to  buy  more  complex  and  sophisticated  weapons  that  cost 
more  money  to  develop  and  produce,  and  has  lead  to  a  higher  per  unit  price  tag  and  the 
purchase  of  smaller,  inefficient  quantities.  The  DoD  continues  to  face  Congressional 
scrutiny  of  its  acquisition  programs  because  of  recurring  problems  with  weapon  system 
acquisition.  54  The  DoD  must  develop  an  approach  to  weapon  system  development  and 
acquisition  that  produces  superior  weapons  while  maintaining  the  defense  industrial  base 
and  keeping  the  trust  and  confidence  of  the  U.S.  taxpayer.  The  use  of  Critical  Success 
Factors  can  help  toward  accomplishing  this  goal. 

From  the  data  that  was  gathered  regarding  Critical  Success  Factors  it  is  clear  that 
while  program  management  by  definition  is  a  one  time  undertaking  that  produces  a 
unique  and  specific  product,  the  systems  engineering  process  that  program  management 
is  based  on  has  enough  structure  to  allow  the  use  of  Critical  Success  Factors  to  identify 
possible  problems  within  a  program. 

The  goal  of  this  research  was  to  answer  the  question: 

•  What  are  the  Critical  Success  Factors  (CSF)  that  influence  the  successful 
acquisition  of  DOD  weapon  system  programs? 

From  the  data  that  was  collected  and  the  analysis  that  was  done,  it  is  clear  that 
there  are  several  Critical  Success  Factors  that  can  help  determine  the  success  of  DoD 
weapon  system  acquisition  programs.  These  four  factors  are: 

•  Stable  and  Adequate  Funding 

•  Clearly  Defined  and  Stable  Requirements 

•  Risk  Management 

•  Technically  Competent  Program  Staff 

While  there  are  many  factors  that  can  affect  a  program  positively  it  is  clear  that  these  four 
factors  listed  above  are  seen  by  program  managers,  acquisition  professionals  and  the 

54  www.govexec.com/dailvfed/07Q5.htm.  McCain  Amendment  Would  Increase  Defense  Acquisition 
Oversight,  06  December  2006. 
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Government  Aceountability  Offiee  as  the  key  elements  to  a  suceessful  aequisition 
program.  Dobbins'  and  Donnelly's  researeh  identified  18  Critieal  Success  Factors  related 
to  DoD  acquisition  programs. Many  were  not  addressed  in  any  detail  by  other  authors 
or  the  GAO.  The  four  Critical  Success  Factors  that  are  mentioned  above  have  been 
identified  extensively  in  books,  journals  and  government  documents  detailed  in  Chapter 
III  and  there  is  extensive  information  on  why  these  four  CSFs  are  key  to  the  success  of 
DoD  acquisition  programs.  There  was  insufficient  data  to  conduct  analysis  on  the  other 
14  Critical  Success  Factors  identified  by  Dobbins  and  Donnelly. 

To  help  answer  the  primary  research  question  the  following  secondary  questions 
were  also  asked,  and  answered: 

•  How  do  we  define  the  success  of  an  acquisition  program? 

As  stated  in  Chapter  III,  the  way  an  organization  defines  success  often  depends  on 
where  that  organization  is  in  the  acquisition  process.  For  example,  the  user's  definition  of 
success  does  not  necessarily  coincide  with  the  contractor's  definition  and  vice  versa.^^  It 
is  also  apparent  that  each  organization  uses  its  own  definition  of  success  and  from  the 
research  done  there  has  not  been  one  definition  of  success  that  is  accepted  by  all  the 
stakeholders. 

B.  RECOMMENDATIONS 

•  The  DoD  must  recognize  that  Critical  Success  Factors  exist  and  are  the 
primary  reason  why  programs  succeed.  The  DoD  must  formally  identify 
the  Critical  Success  Factors  that  apply  to  DoD  acquisition  programs  and 
implement  a  program  to  analyze  and  measure  these  factors  in  order  to 
improve  the  success  of  future  programs. 

•  The  DoD  must  institutionalize  the  lessons  on  Critical  Success  Factors  and 
ensure  that  future  program  managers  understand  the  process  and  can 


James  H.  Dobbins.  &  Richard  G.  Donnelly,  Summary  Research  Report  on  Critical  Success  Factors 
in  Federal  Government  Program  Management,  Acquisition  Review  Quarterly  Winter-98,  1998,  62. 

Defense  Acquisition  University,  Introduction  to  Defense  Acquisition  Management,  1999,  7. 
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successfully  implement  the  program.  This  requires  "buy  in"  from  senior 
aequisition  exeeutives  that  support  and  understand  the  importance  of 
Critical  Success  Faetors. 

•  The  formal  education  and  training  of  program  managers  through  the 
Defense  Aequisition  University  must  include  the  proeess  of  identifying 
Critieal  Suceess  Faetors  and  implementing  the  measures  to  ensure 
sueeessful  programs.  It  is  not  enough  to  understand  what  eritieal  faetors 
are,  there  must  be  a  systematie  proeess  for  identifying  and  measuring 
faetors  that  are  key  to  the  sueeess  of  the  aequisition  program. 

•  The  DoD,  the  Congress,  the  user  and  the  aequisition  eommunity  must 
agree  to  the  definition  of  what  a  sueeessful  program  is.  Until  the  major 
stakeholders  ean  agree  to  what  sueeess  means  and  share  a  eommon  vision 
then  future  programs  will  continue  to  have  major  problems. 

C.  AREAS  OF  FURTHER  RESEARCH 

There  are  two  areas  that  further  researeh  should  explore.  First,  it  is  elear  that  more 
researeh  must  be  done  in  the  area  of  Critieal  Sueeess  Faetors.  There  must  be  a  eoncerted 
effort  to  identify  and  measure  the  effeets  of  Critieal  Sueeess  Faetors  on  DoD  aequisition 
programs.  There  is  not  enough  data  to  validate  the  next  14  Critieal  Sueeess  Faetors  that 
were  identified  by  Dobbins  and  Donnelly,  so  more  researeh  need  to  be  done  in  this 
particular  area.  Further  research  should  focus  on  documenting  the  effeet  that  the  14 
Critieal  Sueeess  Faetors  have  had  on  past  programs  and  possible  predietions  on  future 
programs. 

The  second  area  that  needs  to  be  explored  is  the  relationship  between  internal  and 
external  faetors  and  the  impacts  that  the  two  have  on  programs.  This  researeh  foeused  on 
internal  faetors  that,  for  the  most  part,  eould  be  controlled  within  DoD,  future  researeh 
needs  to  foeus  on  the  relationship  between  outside  faetors  and  program  sueeess,  to 
inelude  the  budget  proeess  and  how  it  ereates  instability  within  programs. 

The  DoD  aequisition  proeess  produees  the  most  teehnically  advaneed  and  lethal 
weapon  systems  in  the  world,  and  our  military  is  unmatehed  in  its  capability  to  project 
lethal  forees  around  the  world.  This  reality  must  be  tempered  with  the  understanding  that 
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defense  budgets  are  eoming  under  greater  scrutiny  and  control  and  as  our  weapon 
systems  become  more  complex  we  buy  fewer  of  them.  The  DoD  must  become  better  at 
producing  weapons  in  a  cost  and  time  constrained  environment.  If  the  DoD  can  adopt  a 
basic  guideline  for  using  Critical  Success  Factors  to  guide  a  program  to  completion  it  will 
streamline  the  process  and  create  a  more  efficient  process  for  weapon  systems 
acquisition. 
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